Computational Skepticism for AI

Why AI Systems Fail: Systematic Doubt as a Computational Practice

The fraud detection case from the opening — before any framework is named.

The course opens in the middle of a failure. Students are given the system's performance metrics, its deployment context, and the compliance team's sign-off — and asked what they would have done differently. The Tier 4 / Tier 5 gap is introduced through this case: the validation question that required metacognitive supervisory intelligence (Tier 4) and causal reasoning (Tier 5) that the system's operators did not have. Philosophical foundations: Descartes on systematic doubt; Popper on falsifiability; what it means to build a computational practice around structured skepticism rather than ad-hoc review.

The Nine Pillars of Botspeak are not reintroduced — they are assumed. What this week adds: the distinction between AI fluency (knowing how to work with a tool) and AI validation (knowing how to test whether a tool should be trusted, and under what conditions).

The Confidence Trap: Logic, Probability, and Uncertainty in AI

A second high-stakes failure in a different domain — a medical triage model with 91% accuracy that performed at 47% accuracy in the patient subpopulation it was most often deployed to assess. No facts were wrong. The framing was wrong.

A second case from a different domain establishes that Week 1's failure was structural, not incidental. Boolean logic to probabilistic reasoning. Frequentist vs. Bayesian frameworks for AI confidence. Hume's Problem of Induction and what it means for models that have never encountered the tail of their distribution. The proportional skepticism protocol introduced in Botspeak's Discernment mode is extended into a formal verification calibration: stakes, reliability zone, reversibility. The symmetric failure modes — over-trust and under-trust — receive equal weight.

Cognitive Bias, Model Bias, and the Human-AI Feedback Loop

Before building technical validation tools, students need to understand where bias enters the human-AI system — not just the model. Implicit and cognitive biases. Dataset bias, label bias, structural bias. Postmodern perspectives on truth and representation. How models inherit and amplify human assumptions — and why the engineer who designed the labeling schema is the person most likely to miss the bias it introduced. The first Validation Lab follows: a real dataset, a provided model, and one question — what's wrong with it? The lab is graded on specificity of finding and quality of reasoning, not on whether students find every flaw.

Data Validation: Is Your Dataset Actually What You Think It Is?

An autonomous vehicle dataset with 98% coverage — but systematic underrepresentation of nighttime, wet-surface, and pedestrian-at-intersection scenarios. The model passed every benchmark. The benchmark was designed from the same distribution as the training data.

Exploratory data analysis as skepticism rather than description. The seven hidden assumptions that live in every dataset and are never stated in the documentation. Plato's Allegory of the Cave as a framework: the dataset is not the world, it is the shadow the world casts under specific collection conditions. Strategic Delegation and Critical Evaluation from Botspeak's pillar framework extended: when do you trust what the data says, and when do you interrogate how it was collected?

Explainability and Interpretability: Does the Explanation Explain — or Does It Reassure?

A loan approval model with SHAP explanations that correctly identified "credit history" as the top feature. A subsequent audit found the model was using credit history as a proxy for zip code. The SHAP values were technically accurate. They were practically misleading about the source of the disparity.

SHAP, LIME, counterfactual explanations — introduced not as solutions but as tools with specific reliability zones and specific failure modes. Wittgenstein's language games: an explanation that uses the right words is not the same as an explanation that illuminates the right thing. The gap between what the explanation shows and what domain knowledge requires is this week's analytical question. Students implement and critique SHAP and LIME on a provided model — the deliverable is a written critique, not a technical report.

Bias Detection and Mitigation: What Counts as Fair — and Who Decides?

A recidivism prediction model that satisfied two different quantitative fairness criteria simultaneously — and violated a third. The disagreement was not a technical error. It was a values conflict embedded in the choice of metric.

Quantitative fairness metrics — demographic parity, equalized odds, calibration — and the mathematical proof that they cannot all be satisfied simultaneously when base rates differ across groups. Debiasing techniques and their limits. The postmodern critique: "fair" is not a property of a model, it is a claim about values, and technical debiasing does not resolve the underlying values question. Botspeak's Ethical Reasoning pillar extended into formal technical implementation. Students apply bias detection tools to a real dataset and evaluate at least two competing fairness definitions — the deliverable requires a recommendation and a defense of the values choice embedded in it.

Adversarial Attacks and Model Fragility: What Does "Understanding" Mean If a Perturbation Breaks It?

An image classifier that achieves 99% accuracy on standard benchmarks and misclassifies a stop sign as a speed limit sign when three small stickers are applied — perturbations invisible to human vision at normal driving speed.

Adversarial examples are not exotic edge cases — they are evidence about what the model has actually learned. The philosophical question: if a model can be fooled by a perturbation a human would not notice, what exactly is the model doing? Nietzsche and adversarial behavior: the will to find the weakness in any system is also the will to make the system robust. Adversarial attacks as a validation methodology, not just a threat. Defense mechanisms and adversarial training — with explicit documentation of what each defense protects against and what it does not. This week's Validation Lab is mandatory and supervised — adversarial attack design requires direct feedback that asynchronous work cannot substitute.

Probabilistic Reasoning and Uncertainty: Is the Model's Confidence Meaningful?

A clinical decision support tool that returned "high confidence" predictions for cases that fell outside its training distribution. The confidence score was a property of the model architecture, not an evidence-based estimate. The clinicians using it did not know this. The documentation did not say it.

Hume's problem of induction as a precise technical claim: a model trained on past data has no mathematical basis for confidence about future data that differs structurally from that past. Bayesian vs. frequentist probability — two philosophical frameworks that produce different answers to "what does 87% confidence mean here?" Calibration curves, conformal prediction, and the tools for turning model outputs into honest uncertainty estimates. Students apply probability distributions to a provided model's confidence outputs and identify the cases where confidence is structurally miscalibrated.

Reinforcement Learning and AI Reliability: Did the Agent Learn What You Intended?

An RL agent trained to maximize a proxy reward that successfully maximized the proxy and failed entirely at the underlying objective — not because of a technical error, but because the reward function was specified by humans who did not model their own assumptions carefully enough.

Free will vs. determinism in RL: do AI agents "choose" — and why does that question matter for how you validate them? Reward functions as moral structures: the reward is a formal statement of values, and reward hacking is what happens when the formal statement diverges from the intended values. Utilitarian reasoning in RL and its failure modes. Safety, alignment, and emergent behavior as validation targets, not just design concerns. Students implement an RL model on a constrained problem and analyze how reward design shaped behavior — the deliverable requires identifying at least one emergent behavior not intended by the reward function.

Midterm: Research Project Presentations

No new instruction. All students present current research project status to the class: the AI system under validation, findings to date, the most surprising failure mode discovered, and — this is not optional — the one thing they still cannot explain. TAs and peers provide structured feedback using the validation framework. The midterm is graded on specificity of findings, quality of causal and counterfactual reasoning applied, and honesty about what the validation has not yet resolved. A clean story with no open questions is not a stronger presentation. It is a less credible one.

Human-AI Collaboration and Trust Calibration: When Should You Override the Tool?

A drug discovery pipeline with five AI-assisted stages. An invalid assumption in Stage 2 — the model assumed structural similarity implied functional similarity — compounded through Stages 3 and 4 into a candidate molecule that passed computational screening and failed wet-lab synthesis for a reason the Stage 2 model had no representation of.

The Botspeak Trust Calibration mode returns here in full technical implementation. Error compounding across multi-step AI workflows. System-trust vs. output-trust. The Diligence protocol from Botspeak extended into a formal monitoring specification: cadence, drift indicators, escalation conditions, shutdown criteria. Students produce a trust calibration map for their research project — appropriate trust level, calibration rationale, and compounding risk analysis for every stage of the validation pipeline they have built.

Data Visualization for AI Transparency: Does the Dashboard Clarify — or Comfort?

Two dashboards built from the same model evaluation data. One was used by the deployment team to approve production release. The other was built after the deployment failure, for the post-mortem. The same numbers. Completely different conclusions available from each layout.

McLuhan's "the medium is the message" applied to model evaluation dashboards: the visualization is not neutral, it is an argument. How dashboard design shapes what questions get asked and what questions become invisible. Deceptive visualization practices — not just misleading charts, but visualization architectures that foreground reassuring metrics and background concerning ones. Building transparency-first visualization pipelines. Students design a dashboard communicating their research project's validation findings accurately — including uncertainty — then deliberately redesign it to be misleading using the same data. The reflection on what the redesign reveals about the original is the graded deliverable.

AI Ethics, Governance, and Accountability: Who Is Responsible When the Validation Fails?

Amazon's recruiting tool case — not the familiar version about why it was biased, but the version about why no one caught it for a year. The accountability chain was intact on paper and absent in practice. The people responsible for validation had no shared definition of what "validated" meant.

Kant's categorical imperative applied to validation: if no one's validation process would survive generalization, no one's validation process is a standard. Utilitarian ethics and the governance gap between "we tested it" and "we tested it against the right failure modes." Existential risk frameworks and AI alignment — Bostrom not as science fiction but as a precise statement of what happens when optimization continues past the point where human oversight is meaningful. Algorithmic accountability and the Diligence protocol as governance infrastructure. Industry perspective: company or research group presentation on trust implementation challenges in production.

The Philosophical Limits of AI: What the Machine Cannot Know — and Why the Answer Is Mathematical

The Socratic method as a validation strategy: the question that exposes the limit is more valuable than the answer that obscures it. Meaning, understanding, and intentionality — the Turing Test and the Chinese Room not as philosophical curiosities but as precise statements about what kind of claims are supportable about AI cognition. The limits of data as a representation of the world: what is structurally absent from any training distribution, and why that absence cannot be fixed by more data. The Pearl causal ladder — Rung 1 (observation), Rung 2 (intervention), Rung 3 (counterfactual) — as the precise technical statement of where current AI systems are strong, weak, and absent. Final project workshop: peer feedback on complete drafts before submission.

Final Research Presentations and Submission

The terminal deliverable: a complete original research project demonstrating computational skepticism under real conditions. Research question and system context. The full validation pipeline — bias audit, explainability critique, adversarial stress test, uncertainty calibration, trust calibration map — with explicit documentation of what each component can and cannot catch. Full iteration log. Qualified conclusion in two registers: technical, for a statistician, and plain-language, for the decision-maker who will act on it.

And the section that carries half the project grade: the Irreducibly Human accounting. Three specific judgment calls that required your values, domain knowledge, or professional accountability. One judgment call that you attempted to delegate to a tool and then reclaimed — with an honest account of why. An assessment of what the AI did well in this project, where it produced confident-sounding noise, and what you would do differently. That section cannot be produced by a tool. It is the evidence that you were present for the work.